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Abstract 

Background: There are currently no reliable markers of acute domoic acid toxicosis (DAT) for California sea lions. 
We investigated whether patterns of serum peptides could diagnose acute DAT. Serum peptides were analyzed by 
MALDI-TOF mass spectrometry from 107 sea lions (acute DAT n = 34; non-DAT n = 73). Artificial neural networks 
(ANN) were trained using MALDI-TOF data. Individual peaks and neural networks were qualified using an 
independent test set (n = 20). 

Results: No single peak was a good classifier of acute DAT, and ANN models were the best predictors of acute 
DAT. Performance measures for a single median ANN were: sensitivity, 100%; specificity, 60%; positive predictive 
value, 71%; negative predictive value, 100%. When 101 ANNs were combined and allowed to vote for the 
outcome, the performance measures were: sensitivity, 30%; specificity, 100%; positive predictive value, 100%; 
negative predictive value, 59%. 

Conclusions: These results suggest that MALDI-TOF peptide profiling and neural networks can perform either as a 
highly sensitive (100% negative predictive value) or a highly specific (100% positive predictive value) diagnostic 
tool for acute DAT. This also suggests that machine learning directed by populations of predictive models offer the 
ability to modulate the predictive effort into a specific type of error. 
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Background 

The toxicosis associated with domoic acid (DA) inges- 
tion has been linked to massive marine mammal strand- 
ing events along the coastal areas of the Western United 
States [1]. The California sea lion (Zalophus california- 
nus) appears to be the most commonly affected species, 
likely due to its foraging on fish containing domoic acid 
[2]. Of sea lions admitted to The Marine Mammal Cen- 
ter (TMMC; Sausalito, CA) between 1998 and 2006 
more than 24% were diagnosed with domoic acid toxi- 
cosis (DAT), of which 77% were acute DAT [3]. Mortal- 
ity was significant, regardless of acute or chronic 
diagnosis, exceeding 40%. 
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Arriving at a diagnosis of DAT is not always straight 
forward. Clinical signs can be highly variable [4] and 
diagnosis often requires post mortem information 
derived from histological examination of the brain. 
Some of the variability in clinical signs in stranded ani- 
mals is likely due to variation in ingested dose, the time 
at which the sea lion strands, and the time at which 
examination occurs during the course of the intoxica- 
tion. Clinical suspicion of DAT is based on the observa- 
tion of abnormal behavior, e.g., ataxia, constant head 
weaving or seizures [4], which may or may not present 
at the time of stranding. The diagnosis solely based 
upon the detection of DA in body fluids or tissues can 
be misleading because (i) DA is rapidly cleared from the 
body in experimental animal models [5] and sea lions 
[6], (ii) the time between ingestion and stranding is 
unknown, (iii) there is no established harmful dose for 
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sea lions. Blood chemistry and hematology data do not 
provide information to allow a definitive diagnosis of 
DAT. High hematocrit, eosinophil counts and serum 
creatine kinase activities have been reported for sea 
lions with DAT [4]; however, only creatine kinase activ- 
ity was found to be abnormal compared with established 
normal parameters for marine mammals. In a larger 
study of 715 sea lion cases from 1998 to 2006, abnormal 
hematological parameters were not found [3] raising the 
question of utility of these parameters in the differential 
diagnosis. 

Diagnosis of DAT is supported by the finding, after 
necropsy, of hippocampal atrophy and neuronal necrosis 
[3,7], although, these changes are not always observed in 
acutely affected animals. Neuropathologic changes in 
hippocampal volume measured by magnetic resonance 
imaging (MRI) or abnormal oscillations in brain electri- 
cal activity measured by electroencephalography (EEG) 
both offer less invasive means to support a diagnosis of 
chronic DAT [3]. Whether or not MRI or EEG can suf- 
ficiently discriminate between living sea lions with acute 
DAT and stranded sea lions without DAT has not been 
determined. Regardless, both MRI and EEG are expen- 
sive, time consuming, and impractical as a diagnosis 
tool during mass stranding events [8]. High-throughput, 
less invasive clinical diagnostic tools for DAT, beyond 
non-specific changes such as abnormal behavior, which 
do not require sedation or post-mortem analysis are 
unavailable. Recently, a semi-quantitative test for habi- 
tuation to external stimuli was reported to classify sea 
lions afflicted with DAT with 50% sensitivity and 93% 
specificity which equates to 81% negative predictive 
value, and 75% positive predictive value [8]. Although 
not perfect, a behavioral test such as this does not nega- 
tively impact the health of the sea lion and provides 
support for the diagnosis of DAT. 

Time-of-flight mass spectrometry has been used as a 
clinical discovery tool for a number of human diseases 
with primary emphasis in cancer [9], but also to classify 
immunological diseases [10,11], response to erythropoie- 
tin therapy [12], renal disease [13], and neurological dis- 
orders [14,15] to name a few. The low molecular weight 
proteome or peptidome found in biological fluids is 
diverse and changes in the profile can be employed to 
discriminate between patients with or without disease 
[16]. In addition, combinations of candidate proteins/ 
peptides can often create a more robust test when indi- 
vidual proteins/peptides fail to discriminate between two 
groups [17-19]. Large transcriptomic datasets analyzed 
with machine learning tools were recently shown to 
enhance the ability to discriminate health status of Cali- 
fornia sea lions [20]. Because clinical signs and hemato- 
logical parameters offer little value towards the 
diagnosis of DAT in sea lions, we investigated whether 



patterns of serum peptides could offer additional sup- 
port to the diagnosis of DAT using a similar machine 
learning approach. 

Results 

Comparison of haematological and clinical parameters 

Sera from stranded sea lions collected at TMMC were 
distributed across multiple years between 2005 and 2010 
in proportions equal to 11.5% (n = 10), 6.9% (n = 6), 
16.1% (n = 14), 17.2% (n = 15), 35.6% (n = 31) and 
12.6% (n = 11), from 2005 to 2010 respectively. Sera 
from the MMP were collected between 2000 and 2008, 
and at the time of sampling 7 of 20 exhibited clinical 
signs which included regurgitation, diarrhea, lameness, 
vesicles, and abscess from bite wound. Only one animal 
had prior seizures but clinical signs were unrelated to 
symptoms associated with DAT. Of the MMP sea lions, 
15 of 20 were fasted at the time of collection and 8 of 
20 blood samples were taken under anesthesia. Descrip- 
tive data available for all individuals in the training data- 
set is summarized in Additional file 1: Table SI. 
Individuals in the stranded non-DAT group reflect etiol- 
ogies common to sea lions admitted to TMMC between 
1991 and 2000 [21]; of these 3,379 non-DAT individuals 
the most prevalent etiologies were malnutrition (35%), 
leptospirosis (30%), trauma (19%), and miscellaneous 
(11%) with carcinoma present in 3% of cases [21]. In 
addition to the 107 animals in the training set, we used 
an independent test set of 20 animals for qualification, 
and the identities and diagnoses were blinded to the pri- 
mary investigator until after analysis. These 20 animals 
from 2007 to 2010 were chosen by TMMC to include 
10 acute-DAT and 10 stranded non-DAT, and in gen- 
eral reflect the types of cases seen in the stranding 
population. Descriptive data available for individuals in 
the test set is summarized in Additional file 1: Table S2. 

Haematological and serum biochemistry data were 
available for serum collected from 76 of the 87 indivi- 
duals in the DAT and stranded non-DAT groups (Table 
1) and all 20 individuals in the managed non-DAT 
group (See Additional file 1: Table S3). Individuals in 
the acute DAT group had significantly higher red blood 
cell counts, hemoglobin levels, and hematocrits com- 
pared to the stranded non-DAT group or the combined 
non-DAT group (including MMP sea lions), despite sig- 
nificantly lower levels of BUN and BUN/creatinine 
ratios (Table 1.). White blood cell counts in the DAT 
group were 1.7-fold lower compared to the stranded 
non-DAT group, but this difference was not significant 
when compared to the combined non-DAT group (p > 
0.05). Likewise banded neutrophils and lymphocytes 
were significantly different in the DAT group relative to 
the stranded non-DAT group (2.9-fold lower and 1.3- 
fold higher, respectively), but this difference was not 
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Table 1 Haematologic and serum biochemistry data of sea lions in the training dataset 



Patients with Blood 
data 


n = 29 
Avg ± S.E. 


n = 47 
Avg ± S.E. 


n = 20 
Avg ± S.E. 


DAT v. Stranded non- 
DAT 




DAT v. Combined 
non-DAT 


Component 


DAT 


Stranded non- 
DAT 


Managed non- 
DAT 


DAT Fold 
Change 


P" 

value 


Combined non- 
DAT 


DAT Fold 
Change 


P" 

value 


WBC 


1 1 .8 ± 0.5 (n = 

29) 


19.6 ± 1.6 (n = 
42) 


6.1 ± 0.6 (n = 
20) 


-1.7 


< 

0.001 * 


15.3 ± 1.3 (n = 
62) 


-1.3 


0.329* 


RBC 


4.9 ± 0.1 (n = 

29) 


4.3 ± 0.1 (n = 

42) 


4.4 ± 0.1 (n = 

20) 


1.1 


< 

0.001 * 


4.3 ± 0.1 (n = 

62) 


1.1 


< 

0.001* 


HGB 


1 7.4 ± 0.5 (n = 

29) 


15.0 ± 0.4 (n = 

42) 


15.9 ± 0.5 (n = 
20) 


1.2 


< 

0.001* 


15.3 ± 0.3 (n = 
62) 


1.1 


< 

0.001* 


HCT 


51.0 ± 1.4 (n = 

29) 


43.9 ±1.1 (n = 
42) 


45.9 ± 1.5 (n = 
20) 


1.2 


< 

0.001* 


44.5 ± 0.9 (n = 
62) 


1.1 


< 

0.001* 


MCV 


97 ± 4.5 (n = 

29) 


103 ± 0.8 (n = 
42) 


104 ± 1.8 (n = 

20) 


-1.1 


0.686* 


103 ± 0.8 (n = 

62) 


-1.1 


0.749* 


MCH 


35 ± 0.3 (n = 

29) 


35 ± 0.3 (n = 

42) 


36 ± 0.7 (n = 
20) 


1.0 


0.930* 


35 ± 0.3 (n = 

62) 


1.0 


0.482 


MCHC 


34 ± 0.2 (n = 

29) 


34 ± 0.2 (n = 
42) 


35 ± 0.2 (n = 
20) 


1.0 


0.582* 


34 ± 0.1 (n = 

62) 


1.0 


0.502 


PLT 


428 ± 28 (n = 
29) 


414 ± 24 (n = 
42) 


358 ± 24 (n = 
20) 


1.0 


0.701 


396 ± 18.0 (n = 
62) 


1.1 


0.323 


RDW 


15.8 ± 0.1 (n = 
20) 


16.2 ± 0.2 (n = 
32) 


15.3 ± 0.4 (n = 
20) 


1.0 


0.054* 


15.9 ± 0.2 (n = 
52) 


1.0 


0.865* 


MPV 


7.7 ± 0.2 (n = 
19) 


8.1 ± 0.2 (n = 

32) 


8.7 ± 0.2 (n = 
20) 


-1.1 


0.072 


8.4 ± 0.1 (n = 
52) 


-1.1 


0.007 


Segs 


6136 ± 173 (n 
= 29) 


6072 ± 345 (n 
= 47) 


3657 ± 367 (n 
= 20) 


1.0 


0.045* 


5351 ± 297 (n = 
67) 


1.1 


0.801* 


Bands 


109 ± 44 (n = 

29) 


311 ± 80 (n = 
47) 


13 ± 13 (n = 
20) 


-2.9 


0.002* 


222 ± 58 (n = 
67) 


-2.0 


0.075* 


Lymphs 


1429 ± 102 (n 
= 29) 


1123 ± 131 (n 
= 47) 


1818 ± 232 (n 
= 20) 


1.3 


0.008* 


1331 ± 121 (n = 
67) 


1.1 


0.097* 


Monos 


231 ± 39 (n = 
29) 


97 ± 18 (n = 
47) 


389 ± 86 (n = 
20) 


2.4 


< 

0.001* 


184 ± 33 (n = 
67) 


1.3 


0.079* 


Eosin 


868 ± 115 (n = 
29) 


247 ± 59 (n = 
47) 


214 ± 62 (n = 
20) 


3.5 


< 

0.001* 


237 ± 45 (n = 
67) 


3.7 


< 

0.001* 


Na 


148 ± 0.9 (n = 
28) 


156 ± 1.6 (n = 

46) 


149 ± 0.5 (n = 
20) 


-1.1 


< 

0.001* 


154 ± 1.2 (n = 

66) 


-1.0 


< 

0.001* 


K 


4.7 ± 0.09 (n = 
28) 


4.3 ± 0.10 (n = 
47) 


4.3 ± 0.10 (n = 
20) 


1.1 


0.026 


4.3 ± 0.08 (n = 
67) 


1.1 


0.012 


CI 


109 ± 0.7 (n = 

28) 


119 ± 1.6 (n = 
47) 


108 ± 0.8 (n = 
20) 


-1.1 


< 

0.001* 


115 ± 1.3 (n = 

67) 


-1.1 


0.002* 


BUN 


21 ± 2.9 (n = 

28) 


100 ± 15.7 (n = 
47) 


26 ± 1 .3 (n = 
20) 


-4.8 


< 

0.001* 


78 ± 11.7 (n = 

67) 


-3.7 


< 

0.001* 


Creatinine 


0.7 ± 0.04 (n = 
28) 


3.1 ± 0.73 (n = 
47) 


1.1 ± 0.08 (n = 
20) 


-4.4 


0.260* 


2.5 ± 0.52 (n = 
67) 


-3.6 


0.016* 


Glucose 


108 ± 4.9 (n = 
28) 


104 ± 4.0 (n = 
47) 


125 ± 5.2 (n = 
20) 


1.0 


0.948* 


110 ± 3.4 (n = 

67) 


1.0 


0.394* 


Ca 


8.3 ± 0.12 (n = 
27) 


8.0 ± 0.13 (n = 
47) 


9.6 ± 0.12 (n = 
20) 


1.0 


0.091* 


8.5 ± 0.14 (n = 
67) 


1.0 


0.405 


P 


5.1 ± 0.3 (n = 

28) 


8.0 ± 0.5 (n = 

47) 


5.7 ± 0.3 (n = 

20) 


-1.6 


< 

0.001* 


7.3 ± 0.4 (n = 

67) 


-1.4 


< 

0.001* 


Total Protein 


7.8 ± 0.15 (n = 
28) 


7.8 ± 0.18 (n = 
47) 


7.3 ± 0.23 (n = 
20) 


1.0 


0.771* 


7.6 ± 0.14 (n = 
67) 


1.0 


0.213* 


Albumin 


3.0 ± 0.1 1 (n = 

28) 


2.4 ± 0.08 (n = 
47) 


3.4 ± 0.05 (n = 
19) 


1.3 


< 

0.001 


2.7 ± 0.08 (n = 
66) 


1.1 


0.370 


Bilirubin Total 


0.5 ± 0.07 (n = 
28) 


0.7 ± 0.18 (n = 
47) 


0.2 ± 0.03 (n = 
20) 


-1.4 


0.774* 


0.5 ± 0.13 (n = 
67) 


1.0 


0.038* 
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Table 1 Haematologic and serum biochemistry data of sea lions in the training dataset (Continued) 



GGT 


116 ± 11 (n = 

27) 


217 ± 25 (n = 
47) 


67 ± 7 (n = 20) 


-1.9 


< 

0.001* 


172 : 
67) 


b 20 (n = 


-1.5 


0.232* 


AST (SGOT) 


65 ± 1 2 (n = 
28) 


62 ± 1 1 (n = 

46) 


21 ± 2 (n = 20) 


1.0 


0.438* 


49 ± 


8 (n = 66) 


1.3 


0.045* 


ALT (SGPT) 


88 ± 13 (n = 
28) 


74 ± 8 (n = 47) 


38 ± 3 (n = 20) 


1.2 


0.095* 


63 ± 


6 (n = 67) 


1.4 


0.002* 


Alkaline Phosphatase 


59 ± 23 (n = 
27) 


50 ± 10 (n = 

47) 


98 ± 9 (n = 20) 


1.2 


0.796* 


64 ± 


8 (n = 67) 


-1.1 


0.081* 


Fe 


86 ± 7 (n = 24) 


94 ± 6 (n = 46) 


115 ± 8 (n = 
20) 


-1.1 


0.470 


100 : 


b 5 (n = 66) 


-1.2 


0.150 


Cholesterol 


172 ± 9 (n = 
28) 


178 ± 10 (n = 

47) 


334 ± 18 (n = 
20) 


1.0 


0.827* 


224 : 
67) 


b 13 (n = 


-1.3 


0.058* 


Triglycerides 


52 ± 4 (n = 28) 


191 ± 35 (n = 

47) 


31 ± 4 (n = 20) 


-3.7 


0.003* 


143 : 
67) 


b 26 (n = 


-2.8 


0.493* 


Mg 


1 .9 ± 0.07 (n = 
23) 


2.5 ± 0.17 (n = 
44) 


2.6 (n = 1) 


-1.3 


0.019* 


2.5 ± 0.17 (n = 
45) 


-1.3 


0.014* 


Creatine Kinase 


1881 ± 417 (n 
= 27) 


1434 ± 571 (n 
= 46) 


341 ± 101 (n = 
20) 


1.3 


0.070* 


1103 
66) 


± 403 (n = 


1.7 


0.003* 


BUN/Cr Ratio 


30 ± 4 (n = 28) 


55 ± 6 (n = 47) 


29 ± 5 (n = 20) 


-1.8 


< 

0.001* 


47 ± 


5 (n = 67) 


-1.6 


0.002* 


Na/K Ratio 


32 ± 0.7 (n = 
28) 


36 ± 1 .4 (n = 

47) 


35 ± 0.7 (n = 
20) 


-1.1 


0.002* 


36 ± 
67) 


1 .0 (n = 


-1.1 


< 

0.001* 


SDH (Sorbitol 
Dehydrogenase) 


21 ± 4 (n = 20) 


30 ± 4 (n = 39) 




-1.4 


0.016* 


n.a. 




n.a. 


n.a. 



A 7"-test (or rank sum test if distribution was non-normal, indicated by '*') was used to compare blood chemistry values between groups 



significant when compared to the combined non-DAT 
group (p > 0.05). Monocytes were 2.4-fold higher and 
eosinophils were 3.5-fold higher in sea lions with DAT 
and both were significantly higher when managed sea 
lions were included in the analysis. Creatine kinase was 
significantly higher in the DAT group than the stranded 
and combined non-DAT groups (1.3 and 1.7-fold 
higher), and sorbitol dehydrogenase was significantly 
lower (1.4-fold) than the stranded non-DAT group with 
no data present for managed non-DAT individuals. 
Levels of Na, CI, P, Mg, and Na/K ratios were signifi- 
cantly lower in the DAT group relative to the stranded 
and combined non-DAT groups (1.1 to 1.0, 1.1, 1.6 to 
1.4, 1.3, and 1.1 -fold lower, respectively), whereas K was 
significantly elevated in the DAT group (1.1-fold). 
Lastly, albumin was significantly higher (1.3-fold) and 
triglycerides were significantly lower (3.7 -fold) in the 
DAT group than the stranded non-DAT group but 
these differences were not significant when managed sea 
lions were included in the analysis (p > 0.05). 

MALDI-TOF profiling 

Processing of MALDI-TOF generated spectra resulted in 
the selection of 104 individual peaks that were used for 
biomarker analysis. To determine if a single peak could 
discriminate between the DAT and non-DAT group, 
receiver operator characteristic (ROC) curves were gen- 
erated using normalized peak height. No single peak 



had area under the curve (AuROC) > 0.8, therefore 
none were deemed excellent classifiers of DAT (Figure 
1). Peaks normalized using Glu-Fib had a mean AuROC 
of 0.543, ranging from 0.383 to 0.692. TIC normalized 
peaks had a mean AuROC of 0.538, ranging from 0.396 




0.5 0.6 0.7 0.8 
AuROC 

Figure 1 Distribution of AuROC for 104 peaks normalized to 
Glu-Fib or TIC Receiver operator characteristic curves were 
constructed using peaks normalized to either Glu-Fib or TIC. Area 
under the ROC curve (AuROC) was calculated for 104 peaks and 
binned every 0.1 ± 0.05. 
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to 0.754. The best individual classifier was peak 3017 ml 
z (TIC normalized) which had an AuROC ± S.E. of 
0.754 ± 0.054 (Figure 2). When the independent test set 
was used to qualify the performance of peak 3017 m/z 
with a threshold determined by minimal-misclassifica- 
tion the test achieved 100% specificity but only 20% sen- 
sitivity with 8 of 10 acute-DAT individuals being called 
incorrectly (Table 2; Additional file 2: Table S4). 

In addition to evaluating individual peak performance 
for predicting DAT, we were also interested in whether 
there were peaks that predicted individuals in the mana- 
ged non-DAT group comprised of sea lions under long- 
term care of the MMP. When performance was evalu- 
ated for discriminating managed non-DAT versus 
stranded individuals, 21 TIC normalized peaks had an 
AuROC > 0.8 and the best performer, 1362 m/z, had an 
AuROC ± S.E. of 0.979 ± 0.023 (see Additional file 3: 
Figure SI). Interestingly this peak was mostly absent in 
stranded sea lion sera. Using the OpT threshold, the 
individuals in the managed non-DAT group were called 
correctly 18 of 20 times (90% sensitivity), and of the two 
mis-called sea lions (#2 and #9), sea lion #2 showed 
clinical signs (behavioral; poor performance). Moreover 
only four stranded individuals (CSL 6896, 9111, 9271 
and 9770) were called incorrectly (95% specificity). 
Using this same threshold only one individual in the 
independent test set was called managed non-DAT 
(CSL 9766, an adult female with acute DAT which had 
recovered and was ultimately released). 

Because no single peak was an excellent classifier of 
DAT, unsupervised hierarchal clustering was used to 




1-specificity (false positives) 

Figure 2 Receiver operator characteristic curve for peak 3017 

m/z. The TIC normalized training dataset was used to generate an 
ROC curve with an AuROC ± S.E. of 0.754 ± 0.054. Four thresholds 
are shown by arrows: minimum mis-classified cut-off (hollow arrow), 
optimum threshold (solid arrow), negative predictive value cut-off 
(striped arrow), and positive predictive value cut-off (grey arrow). 

V / 



determine whether multiple peaks could separate 
DAT from non-DAT sea lions (See Additional file 4: 
Figure S2). The two groups did not form separate 
clusters, suggesting unsupervised methods are not 
capable of discriminating the complex relationship 
between DAT and non-DAT. Therefore peak data 
were modeled using a supervised informatic method: 
feed-forward artificial neural networks (ANNs). Glu- 
Fib and TIC normalized peaks were used separately, 
and 101 ANNs were trained. In addition to using 
individual ANNs, we also combined all 101 ANNs 
and allowed each to vote for the outcome (CANN- 
vote), or simply averaged all 101 predictions 
(CANN 101 ). The generated models were qualified 
using a blinded independent test set of 20 sera from 
stranded sea lions, which were diagnosed with acute 
DAT (n = 10) or non-DAT (n = 10). Using decision 
thresholds determined a priori, the performance of 
each model was evaluated for each of the 20 sea lions 
(Table 2). Compared to the single peak 3017 m/z 
which gave 100% specificity but only 20% sensitivity, 
testing different individual ANNs we achieved high 
specificity (100%) or high sensitivity (100%). Specifi- 
cally, we found the best performance of Glu-Fib nor- 
malized data was 30% sensitivity and 100% specificity, 
which was achieved using a median ANN (Glu-Fib- 
ANN 53 ). Relative to ANNs trained on Glu-Fib nor- 
malized data, models trained using TIC normalized 
data achieved higher sensitivity (100% versus 40%) as 
well as high specificity (90%; Supporting information 
Tables S5 and S6). A negative predictive value of 
100% was achieved using a median ANN (TIC-ANN^ 
which was the highest seen in any model (Table 2). 
This model predicted all 10 DAT individuals correctly 
with four false positives. The four individuals that 
were predicted incorrectly could not be explained by 
sex, age, primary etiology or blood chemistry (See 
Additional file 2: Table S4). It was also interesting to 
note that peak 3017 m/z was a large contributor to 
ANNs trained on TIC normalized data (Figure 3). We 
also observed that other median TIC ANNs (i.e., 
ANNs with AuROCs equal to the median) had differ- 
ent performance measures despite the same AuROC 
(e.g., TIC-ANNi versus TIC-ANN 67 ; Table 2), and 
overall using the OpT threshold when different from 
the minMC threshold resulted in higher sensitivity 
with minimum loss of specificity. For example, in the 
case of TIC-ANN 6 7, the OpT improved performance 
while maintaining the same specificity as the minMC 
(Table 2). When allowed to vote, we achieved 100% 
specificity and 30% sensitivity (Glu-Fib-CANN-vote; 
Table 2), while other CANN models did not perform 
as well when 100% specificity was achieved (Support- 
ing Information Tables S5 and S6). 
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Table 2 Qualified performance of models using an independent test set 







3017 m/z 


ANN n 


ANN 67 


ANN 67 


ANN 53 


CANN-Vote 




Normalization 


TIC 


TIC 


TIC 


TIC 


Glu-Fib 


Glu-Fib 




AuROC 


0.7538 


0.9412 


0.9412 


0.9412 


0.9428 


na 




Threshold 0 


minMC 


OpT 


OpT 


minMC 


OpT 


> 50 votes 


Test Set 


sensitivity 


0.20 


1.00 


0.40 


0.70 


0.30 


0.30 




specificity 


1.00 


0.60 


0.90 


0.90 


1.00 


1.00 




PPV 


1.00 


0.71 


0.80 


0.88 


1.00 


1.00 




NPV 


0.56 


1.00 


0.60 


0.75 


0.59 


0.59 


Training Set 


sensitivity 


0.35 


0.91 


0.88 


0.91 


0.94 


1.00 




specificity 


0.93 


0.95 


1.00 


0.99 


0.96 


0.99 




PPV 


0.71 


0.89 


1.00 


0.97 


0.91 


0.97 




NPV 


0.76 


0.96 


0.95 


0.96 


0.97 


1.00 



a Thresholds calculated a priori based on training set data. minMC, minimum mis-classified threshold; OpT, optimum threshold (OpT). 
Performance against training set is given for comparison 



Discussion 

In this study we show patterns of serum peptides can be 
used to discriminate between stranded sea lions afflicted 
with acute DAT and those unaffected, with excellent 
positive or negative predictive value (i.e., 100%). Cur- 
rently DAT is diagnosed based on a variety of clinical 
signs [3,4,22] as well as the presence of domoic acid in 



body fluids [1,4], but is generally only confirmed post- 
mortem [7,8,22]. Moreover, individuals are further char- 
acterized along a continuum of toxicosis ranging from 
acute to chronic, with the former recovering over time 
and the latter progressing to status epilepticus [3,4,22]. 
The symptomology separating acute from chronic DAT 
can be inconclusive, such as inconsistent MRI- 



1018' 
1118 5 




3017 
3458 



0 10 20 30 40 50 60 70 80 90 100 

ANN (number) 

Figure 3 Relative contribution of each MALDI-TOF peak to each of the 101 ANNs trained using TIC normalized data. The peaks with the 
10 highest average percent contribution are indicated, with superscripted rank. Peak 1362 m/z is indicated for reference. 
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observable hippocampal atrophy associated with chronic 
DAT [3,7,22,23]. As a diagnostic tool MRI is expensive, 
requires sedation and is impractical during a mass 
stranding event [8]. Overall, the lack of a clear gold- 
standard for diagnosing DAT highlights the complexity 
of the underlying physiopathology. Additionally, indivi- 
duals on the far end of the spectrum, chronic DAT, are 
poor rehabilitation candidates and often euthanized [22]. 
Therefore the best test would be one that is rapid, accu- 
rate (minimizing false positives), and high- throughput to 
allow for immediate testing even in the absence of 
abnormal behavior. 

Peptide profiling by MALDI-TOF is a high- throughput 
tool which is sensitive across a large dynamic range [24] 
and has been used to classify diseases in humans [25]. A 
major concern with any biomarker study is experimental 
design [18], therefore we focused on three areas of likely 
variability: DAT diagnosis, sample handling, and case- 
controls. For our training set we used sera from indivi- 
duals that were diagnosed with acute DAT (according 
to [3,22]) without confounding etiologies and rejected 
non-DAT individuals with signs of DAT (acute or 
chronic). We also chose to not use pregnant females 
since we assumed their blood peptide levels would be 
greatly influenced by pregnancy. Pregnant females are a 
key group to be addressed in future research since stu- 
dies suggest pregnant females are spatially more likely 
to be exposed to domoic acid [26] and approximately 
25% of females admitted between 1998 and 2006 had 
evidence of reproductive failure [3]. To account for 
technical variability due to serum handling, specifically 
the effect of serum clotting time on peptide profiles 
[19,27], only sera banked at -80°C on the day of collec- 
tion were utilized. In addition, sea lions were frequency 
matched to control for differences in age and sex. Hae- 
matologic comparisons of stranded sea lions with acute 
DAT and those without showed that individuals with 
acute DAT have significantly increased hematocrit, eosi- 
nophil counts, and levels of creatine kinase in congru- 
ence with previous reports [4,28]. Higher hematocrit 
may indicate dehydration although individuals with 
acute DAT had significantly lower BUN and BUN/crea- 
tinine ratios that aligned with managed non-DAT indivi- 
duals suggesting other influences aside from dehydration 
could affect hematocrit. The clinical use of BUN and 
BUN/creatinine ratio in the context of domoic acid toxi- 
cosis has not been well described, but monosodium glu- 
tamate administration in rats has been shown to 
decrease serum BUN and creatinine while elevating 
BUN/creatinine ratio [29]. We also found that acute 
DAT individuals had a 1.7 to 1.3 -fold reduction in white 
blood cell counts compared to non-DAT individuals, 
though these values are still within reported normal 
ranges (9.4 to 22.8; [30]). This is contrary to reports of 



dolphins exposed to DA in which there was a concur- 
rent eosinophil and white blood cell increase [31]. 
Although haematology and serum biochemistry are use- 
ful for narrowing the diagnosis, alone they cannot con- 
firm specific etiologies. We postulate that these broad 
differences may indicate an underlying pathology speci- 
fic to domoic acid exposure {e.g., neuronal necrosis and 
eosinophilia). 

Using MALDI-TOF we selected 104 peaks in the 
training set of 107 individuals to be used for calculating 
different thresholds a priori for single markers and 
models based on different performance goals, e.g., high 
specificity while minimizing false positives. Performance 
was qualified using an independent test set since it is 
known that cross-validation alone overestimates model 
performance [18]. Since these peaks reflect a fraction of 
the peptidome and any changes reflect changes in the 
body due to the nature of blood circulation, we 
hypothesized that there would be a peptide or group of 
peptides that could discriminate sea lions with DAT, 
but no single peak was a good discriminator of DAT 
(AuROC > 0.8). Interestingly, individuals in the mana- 
ged non-DAT group could be largely classified based on 
the absence of peak 1362 m/z with only two managed 
individuals being misclassified. Furthermore, peak 1362 
m/z was able to largely discriminate the stranded popu- 
lation from the managed population, with only five 
stranded individuals being classified incorrectly. Because 
the managed population is comprised solely of male sea 
lions it is possible that 1362 m/z is a sex specific mar- 
ker; however four out of five misclassified sea lions from 
the stranding population were female thereby refuting 
this idea. While we contend that this peak may be a 
marker of stress it is also possible that differences in 
sample handling may also influence the presence of this 
peak [19,27]. Given the differences in peak 1362 m/z 
between the managed and stranded populations we feel 
additional studies are warranted to determine the iden- 
tity and utility of this peptide. 

Because no single peak was an excellent classifier of 
DAT or non-DAT, neural network models were con- 
structed to interrogate multidimensional relationships. 
Previous reports have demonstrated that more than one 
classifier can increase test performance, because a panel 
of biomarkers is more robust to inherent biological per- 
turbation and individual variation is mitigated by the 
group [19,20,32]. Similarly, we found that artificial 
neural networks trained using all 104 peaks outper- 
formed single classifiers based on increased sensitivity. 
The model that gave the highest sensitivity (100%) and 
specificity (60%), TIC-ANN^ only mis-classified four 
individuals in the test set as DAT. This could not be 
explained by sex, age, primary etiology or blood chemis- 
try, indicating the model is not overtly biased. 
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Interestingly the most weighted peak in resulting ANNs 
was also the best single classifier of DAT (3016 m/z) 
highlighting the potential importance of this peak as a 
classifier. Lastly, our results highlight the importance of 
evaluating different decision thresholds, which can 
improve performance and sensitivity while maintaining 
specificity. This may prove invaluable in diagnosing 
DAT since minimizing false positives and maintaining 
acceptable sensitivity will facilitate diagnosis and 
treatment. 

Conclusions 

Despite the need for a highly specific test for DAT, cur- 
rently there is little information on the accuracy of cur- 
rent DAT diagnostic approaches. We achieved 100% 
sensitivity and 60% specificity in a single model (TIC- 
ANNi) and demonstrated that MALDI-TOF peptide 
profiling and neural networks together can perform as a 
highly sensitive (with 100% NPV) or highly specific 
(with 100% PPV) diagnostic tool for acute DAT. 
Although the test set NPV is 100% for TIC-ANN^ the 
training set NPV at the same threshold suggests that the 
true NPV for this test is likely 96% (Table 2). This tech- 
nique also has the potential to be used as a high- 
throughput diagnostic tool to allow for immediate test- 
ing even in the absence of abnormal behavior, although 
due to the technical nature of the process it is less likely 
to be a point-of-care test and better suited for a centra- 
lized laboratory. Future studies will also address the use- 
fulness of combined peptidomic or proteomic analysis to 
discriminate sea lions with chronic DAT from sea lions 
with acute DAT, which has increased in prevalence in 
recent years [3]. Since samples can be drawn at the time 
of admission, a prognostic test may be developed to 
place individuals in risk categories and/or diagnosis on 
the continuum of acute to chronic DAT based on nega- 
tive outcome of individuals over time. Techniques such 
as these may also provide insight into the biology of 
progression from acute to chronic DAT, or markers of 
DA exposure in individuals without identifiable clinical 
signs. In conclusion, this test highlights the benefits and 
potential of using MALDI-TOF peptide profiling as an 
accurate, rapid, non-invasive, robust tool to identify sea 
lions with acute DAT. 

Methods 

Serum collection and storage 

Samples were collected by The Marine Mammal Center 
(TMMC; Sausalito, CA) under IACUC approved proto- 
cols and under the National Fisheries Service permit 
number 932-1489-00 and by the U.S. Navy Marine 
Mammal Program (MMP; San Diego, CA) as part of a 
standard clinical care regime. At both locations, blood 
samples were collected in serum separator tubes (SST) 



and allowed to clot for 30 to 60 min, centrifuged, serum 
transferred into cryo vials, and stored at -80°C. Less than 
7 h passed between clotting and storage at -80°C. Since 
it has been shown that the number of freeze thaw cycles 
can affect the measured peptide profile of frozen sera 
[33], all samples were thawed at the Medical University 
of South Carolina at 37°C for one min, then placed on 
ice and 60 to 110 (iL aliquots were stored at -80°C. 
Hematology and serum biochemistry data were included 
for comparison if the data corresponded to the same 
day as the drawn sample. 

Inclusion criteria 

Individuals in this study were divided into three groups: 
those suffering from acute domoic acid toxicosis (acute 
DAT), individuals admitted to TMMC that were asymp- 
tomatic for DAT (stranded non-DAT), and individuals 
from the MMP that were asymptomatic for DAT (man- 
aged non-DAT). Except for one sea lion that had just 
been admitted into the MMP from TMMC, all MMP 
individuals had been under the long-term care of veteri- 
narians, hence were collectively deemed 'managed'. The 
remaining two groups, acute DAT and stranded non- 
DAT were sampled at TMMC and were defined using 
available clinical parameters. We retrospectively identi- 
fied serum samples from California sea lions that 
stranded alive along the central California coast between 
2005 and 2010 (n = 2343). Of these, sera drawn within 
seven days of admission to TMMC were allowed 
(-2000). We included sera from both sexes and adult, 
subadult, juvenile, and yearling age classes (as deter- 
mined by [21]). Individuals were included in the acute 
DAT group based on clinical signs such as seizures or 
neurological clinical signs [4] and in some cases the pre- 
sence of domoic acid in bodily fluids. Specifically, acute 
DAT cases were differentiated from individuals with 
chronic DAT as described by Goldstein et al. [3] or with 
available brain histology (hippocampal atrophy indicated 
chronic DAT; [7]). Individuals included in the stranded 
non-DAT group stranded for reasons other than DAT. 

Exclusion criteria 

Sera from known pregnant females (i.e., those that 
aborted later in rehabilitation or with a fetus in uterus 
at necropsy), individuals with significant trauma (e.g., 
missing limbs or life threatening wounds), or individuals 
later diagnosed with chronic DAT were excluded from 
the study. Sera were excluded if collected more than 
seven days after admission, by heart-stick, post-mortem, 
or if they were not archived at -80°C the day of collec- 
tion. Individuals were excluded from the acute DAT 
group if there was a confounding etiology (e.g., carci- 
noma or leptospirosis). Leptospirosis was diagnosed 
based on blood chemistry or at necropsy (as described 
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in [21]). Five individuals in the acute DAT group were 
missing hematology data but had no indications of lep- 
tospirosis (e.g., post-mortem kidney changes characteris- 
tic of leptospirosis [21]). Conversely, individuals were 
excluded from the stranded non-DAT group if seizures/ 
other neurological problems were observed during their 
time in rehabilitation (regardless of etiology), histology 
indicated brain morphological changes consistent with 
DAT, they were positive for DA in bodily fluids, or if 
they later stranded with signs of DAT. 

Experimental design 

A training sample set was analyzed followed by an inde- 
pendent test set used to qualify biomarker and model 
performance. The training set (n = 107) was comprised 
of nearly all acute DAT sera samples available from 
TMMC meeting the criteria described above, frequency 
matched non-DAT sea lions, and non-frequency 
matched managed samples from the MMP (See Addi- 
tional file 1: Table SI). The independent test set (n = 
20) was blinded to Medical University of South Carolina 
personnel processing the samples and analyzing the 
data. Sea lion samples included in the test set were ran- 
domly chosen from the TMMC acute DAT and non- 
DAT sample population prior to analysis of the training 
set. Test set samples were processed and analyzed on a 
day that was different than training set samples. 
Furthermore, the test set was shipped separately from 
the training set to avoid any handling bias between 
training and test set samples. Sera from the training set 
were extracted and analyzed on two different days. To 
limit the effect of interday MALDI-TOF variability [27], 
the training set was divided into two groups (Day 1 or 
Day 2) that had equal sample proportions representing 
acute DAT, stranded non-DAT, and managed sea lions. 
Each group was equally represented for sex, age class, 
draw year, and outcome (release or euthanasia/death). 

MALDI-TOF 

Immediately before being processed, serum aliquots 
were thawed at 37°C for one min after which 50 \iL was 
transferred to a 200 \iL PCR tube. Additional freeze- 
thaws were not allowed. Next, each serum was diluted 
to 0.1% (v/v) TFA using 100 uL of 0.15% (v/v) TFA 
(Thermo Scientific, Rockford, IL) and incubated at room 
temperature for 5 min prior to the addition of 10 \iL 
C8-magnetic beads (ClinProt™ Profiling Kit, Bruker 
Daltonics, Billerica, MA). Magnetic beads and serum 
were incubated for 1 min at room temperature, followed 
by three wash steps of 100 \iL 0.1% (v/v) TFA according 
to manufacturer's guidelines. Peptides were eluted with 
20 \iL of 50% acetonitrile in stabilization buffer (Bruker 
Daltonics) and 15 \iL was transferred to a clean tube. 
Finally, 30 \iL of matrix [5 mg ml/ 1 a-cyano-4- 



hydroxycinnamic acid (Bruker Daltonics) in HPLC grade 
methanol:acetonitrile:water (5:4:1) containing 25 nM 
glu-l-fibrinopeptide peptide mass standard (Glu-Fib; 
Protea Biosciences, Inc., Morgantown, WV)] was added. 
Two (iL of sample matrix was spotted onto a ground 
steel target plate (MTP 384 ground steel T F plate, Bru- 
ker Daltonics). Matrix assisted laser desorption ioniza- 
tion time of flight (MALDI-TOF) spectra were acquired 
using a Bruker AutoflexIII. Each spectra is the sum of 
5,000 shots with the laser moving every 1,000 shots 
across a polygon pattern. A calibration mixture from the 
manufacturer (Bruker Daltonics) was used to calibrate 
the instrument during acquisition (every three to eight 
samples) to correct for spatial and temporal drift. Reso- 
lution was determined using the Glu-Fib internal stan- 
dard and was 3200 with an average mass error of 148 
ppm. 

Data processing and analysis 

Statistics on haematologic and serum biochemistry 
values were performed using SigmaPlot (v. 11.2) to com- 
pare two groups. A T-test was used to determine differ- 
ences between two groups, with normality first being 
evaluated using a Shapiro-Wilk test (a = 0.05). Non- 
normal data was evaluated using a Mann-Whitney Rank 
Sum test (a = 0.05). Raw MALDI-TOF spectra were 
processed using Progenesis MALDI (Nonlinear USA 
Inc., Durham, NC). Spectra were pre-processed using a 
noise filter size of 5 and background subtracted using a 
top hat filter size of 200. Spectra were aligned using a 
search area of 5 and 20 iterative cycles. Peaks with a 
weighted average above 1500 cps were automatically 
selected and manually inspected for inclusion. Peak 
intensities were normalized to the internal standard 
(Glu-Fib) or Total Ion Current (TIC) and analyzed sepa- 
rately based on normalization procedure. If TIC was 
used, Glu-Fib peaks were removed before importing 
into Progenesis MALDI. The independent test set was 
aligned with the training set to ensure proper peak 
alignment. These data were used for downstream ana- 
lyses. Additionally, Matlab (Math Works R2010b, Natick, 
MA) was used to perform unsupervised hierarchical 
clustering analysis of the TIC normalized peak data 
from the training set. 

Receiver operator characteristic (ROC) curve analysis 

Receiver operator characteristic curves were generated 
and the area under the curve (AuROC) was determined 
using Matlab. A 95% confidence interval for each 
AuROC was determined by calculating the standard 
error according to Hanley et al. [34] with a = 0.05. Four 
different thresholds were calculated: minimum mis-clas- 
sified threshold (minMC), optimum threshold (OpT), 
100% negative predictive value (NPV; proportion of 
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predicted negatives which are true negatives) threshold 
(npvT) and 100% positive predictive value (PPV; propor- 
tion of predicted positives which are true positives) 
threshold (ppvT). The minMC is the threshold that 
minimizes the proportion of false-negatives and false- 
positives. The OpT is the geometrically determined 
threshold which is the closest point to 100% sensitivity 
and 100% specificity which corresponds to the perfect 
test [35]. The npvT and ppvT are the thresholds with 
the highest specificity and 100% NPV or the highest 
sensitivity and 100% PPV, respectively. 

Artificial neural network analysis 

The artificial neural network (ANN) algorithm was 
trained using Glu-Fib peptide or TIC normalized peak 
data from the training dataset. These training sets were 
used independently to train 101 feed-forward ANNs 
performed by Matlab as previously described [36]. 
Briefly, each ANN was pursued to an early stop criteria 
determined by internal testing using bootstrapped cross- 
validation with every seventh sample, in addition to 
screening for optimal topology. The median performing 
ANN, determined using AuROC values, were selected 
for qualification with the independent test set. In the 
case where the median ANN shared identical AuROC 
values, both were selected for qualification. It is impor- 
tant to note that, to avoid model over-fitting, a priori 
threshold values determined by ROC analysis of the 
training set were utilized as decision threshold values 
for qualification of the test set. 

In a separate analysis, we determined statistical perfor- 
mance measures for each of the 101 ANNs using the 
test dataset for all thresholds (minMC, OpT, npvT, 
ppvT) and both normalization procedures (TIC and 
Glu-Fib), which resulted in a total of 808 tests. The 
ANN output was averaged across the 101 ANNs for 
each sea lion and was referred to as Combinatorial 
ANN (CANNioi). A priori threshold values for qualifica- 
tion of the test set by CANN 10 i models were derived by 
ROC analysis of the averaged ANN output. In addition 
to CANNioi we developed a scheme to allow each of 
the 101 ANNs to "vote". This was accomplished by con- 
verting the ANN outputs > 0.5 to l's and summing the 
values for each sea lion. If the sum of the 101 ANNs for 
a sea lion was > 50, then that sea lion was predicted to 
be DAT. 

Additional material 



Additional file 1: Table SI. Descriptive data of sea lions in the training 
dataset. Trauma included lacerations, bite wounds, broken bones, and 
anophthalmia. Pneumonia included verminous and/or parafilaroides. 
Table S2. Descriptive data of sea lions in the independent test set. 
Trauma was only to the flipper, and unknown was best described as 
lethargic. Pneumonia included parafilaroides. Empty categories under 



'Primary Etiology' given to serve as a reference to training set descriptive 
data. Table S3. Haematologic and serum biochemistry data of sea lions 
in the independent test set. Blood chemistry data is based on analysis of 
blood drawn the same day as serum samples used for analysis. A 7-test 
(or rank sum test if distribution was non-normal, indicated by '*') was 
used to compare blood chemistry values. 

Additional file 2: Table S4. Qualified performance of peak 3017 m/z 
using an independent test set. DAT is indicated by a '1' and non-DAT a 
'0'. Performance against training set is given for comparison. Table S5. 
Qualified performance of ANNs trained with Glu-Fib normalized data. 
Performance was qualified using an independent test set. DAT is 
indicated by a T and non-DAT a '0'. Performance against training set is 
given for comparison. Table S6. Qualified performance of ANNs trained 
with TIC normalized data. Performance was qualified using an 
independent test set. DAT is indicated by a T and non-DAT a '0'. 
Performance against training set is given for comparison. 

Additional file 3: Figure SI. Peak 1362 m/z spectra, ROC curve and 
performance. (A) 1362 m/z peak height was plotted between two 
groups. The dotted line corresponds to the threshold determined by 
ROC curve analysis. (B) An ROC curve was generated using 107 
individuals in the training set and was used to determine the optimum 
threshold (OpT; indicated by arrow). (C) The OpT was used to determine 
statistical performance measures of peak 1362 m/z using the 107 
individuals in the training set. (D) Statistical performance measures for 
peak 1362 m/z when all sea lions in the study (Training + Test) were 
combined. 

Additional file 4: Figure S2. Unsupervised clustering of MALDI-TOF 
profiles of the training set. Data were standardized by each peak (n = 
104; rows) and represented such that the mean is 0 and standard 
deviation is ± around this mean. Columns are individuals (n = 107), 
labeled at the top with red squares for acute-DAT and blue squares for 
non-DAT. 
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